Assignment 5¶

Q1. Classification Model (40 points)¶

Best Model accuracy: 96.74%

Ground Truth Class Predicted Class Visualization Interpretation
Chair Chair point_cloud_476.gif Correctly classified as chair
Vase Vase point_cloud_700.gif Correctly classified as vase
Lamp Lamp point_cloud_952.gif Correctly classified as lamp
Chair Lamp 406_point_cloud_mis_lamp.gif Wrongly classified as lamp. The reason might be the closed configuration of the chair
Vase Lamp 619_mis_lamp.gif Wrongly classified as lamp. The slender shape of the vase with a single slender flower in it might have confused the model to predict it as a lamp.
Lamp Vase point_cloud_726_miss_vase.gif Lamp wrongly classified as a vase. The reason seems to be a huge round base of the lamp, that kind of looks like a flower pot.

Q2. Segmentation Model (40 points)¶

Best Model accuracy: 89.78%

Ground Truth Segmentation Predicted Segmentation Accuracy (%) Interpretation
gt_42_acc_0.990.gif pred_42.gif 99.0 Near perfect segmentation
gt_48_acc_0.990.gif pred_48.gif 99.0 Near perfect segmentation
gt_113_acc_0.992.gif pred_113.gif 99.2 Near perfect segmentation
gt_255.gif pred_255_acc_0.453.gif 45.3 Failure case: Seat has been combined with the base of the chair. It might be because point coordinates have almost same z dimension that might have caused the blending
gt_272.gif pred_272_acc_0.465.gif 46.5 Failure case: Seat has been combined with the base of the chair. It might be because point coordinates have almost same z dimension that might have caused the blending

Q3. Robustness Analysis (20 points)¶

Ablation: Point cloud rotation¶

Rotation affects both classification and segmentation with almost the same intensity. It changes the coordinates of every point. This doesn't change the local relative information, but the global information changes. Incorporating rotation type of data augmentation would help resolving this issue. In case of segmentation, the model has seen the seats of the training data to be horizontal, hence the results show the red segmentation to be almost horizontal in each case. That means the horizontal bias overpowers the local relative information of the seat of the chair.

Classification¶

0 deg 30 deg 45 deg 60 deg 90 deg
Chair point_cloud_10.gif point_cloud_10-2.gif point_cloud_10-3.gif point_cloud_10-4.gif point_cloud_10_chair_lamp.gif
Lamp point_cloud_730.gif point_cloud_730-2.gif point_cloud_730-3.gif point_cloud_730-4.gif point_cloud_730-5.gif
Vase point_cloud_900.gif point_cloud_900-2.gif point_cloud_900-3.gif point_cloud_900-4.gif point_cloud_900-5.gif
Test Accuracy (%) 96.74 84.89 72.19 64.32 47.64

Segmentation¶

0 deg 30 deg 45 deg 60 deg 90 deg
pred_10.gif pred_10-2.gif pred_10_acc_0.568.gif pred_10-3.gif pred_10_acc_0.353.gif
pred_300.gif pred_300-2.gif pred_300_acc_0.611.gif pred_300-3.gif pred_300_acc_0.379.gif
pred_600.gif pred_600-2.gif pred_600_acc_0.556.gif pred_600-3.gif pred_600_acc_0.240.gif
Test Accuracy (%) 89.78 65.35 54.65 47.65 35.35

Ablation: Number of Points¶

Varying the number of points in the test data doesn't seem to affect the performance of the segmentation and classification models because neither the global or local information is being lost or changed.

Classification¶

100 1000 2000 4000 10000
Chair point_cloud_10-6.gif point_cloud_10-7.gif point_cloud_10-3.gif point_cloud_10-4.gif point_cloud_10-5.gif
Lamp point_cloud_730.gif point_cloud_730-2.gif point_cloud_730-3.gif point_cloud_730-4.gif point_cloud_730-5.gif
Vase point_cloud_900.gif point_cloud_900-2.gif point_cloud_900-3.gif point_cloud_900-4.gif point_cloud_900-5.gif
Test Accuracy (%) 92.44 96.24 96.53 96.64 96.74

Segmentation¶

100 1000 2000 4000 10000
pred_10.gif pred_10-2.gif pred_10_acc_0.837.gif pred_10_acc_0.825.gif pred_10-3.gif
pred_300.gif pred_300-2.gif pred_300_acc_0.945.gif pred_300_acc_0.951.gif pred_300-3.gif
pred_600.gif pred_600-2.gif pred_600_acc_0.982.gif pred_600_acc_0.986.gif pred_600-3.gif
Test Accuracy (%) 79.21 88.79 89.54 89.76 89.78

Q4. Expressive architectures (10 points + 20 bonus points)¶

PointNet++¶

Best Model Accuracy: 97.59%

PointNet++ is an extension of PointNet. It employs heirarchical feature learning from points. In pointnet, each point was handled independently without any kind of grouping. Heirarchical architecture in PointNet++ involves sampling and grouping layers that improves the local and global feature learning. In the earlier layers, local information is extracted out and global information is effectively learned in the later layers.

To implement this, I first did furthest point sampling which covers the maximum span of the point cloud. This gives the potential centroids within the point cloud. Now using a simple KNN algorithm, I grouped the surrounding k (10) points with the centroid. And I just implemented PointNet on these clusters.

Ground Truth Class Predicted Class Visualization Interpretation
Chair Chair point_cloud_476.gif Correctly classified as Chair
Vase Vase point_cloud_700.gif Correctly classified as Vase
Lamp Lamp point_cloud_952.gif Correctly classified as Lamp
Chair Chair point_cloud_2.gif Correctly classified as Chair
Chair Chair point_cloud_406.gif Rectified: Correctly classified as Chair, which was not the case in the vanilla Pointnet
Lamp Vase point_cloud_726.gif Failure Case: Still not able to classify correctly
Vase Lamp point_cloud_619.gif Failure Case: Still not able to classify correctly

Point Transformer¶

Best Model Accuracy: 98.12%

Point Transformer incorporates a self attention mechanism in the point cloud. Each point attends to some k points. These k points are selected by a simple KNN algorithm. Here query is the point itself, and keys and values are from the KNN points. I have taken k to be 10 and point embedding dimension to be 64.

Ground Truth Class Predicted Class Visualization Interpretation
Chair Chair point_cloud_476.gif Correctly classified as Chair
Vase Vase point_cloud_700.gif Correctly classified as Vase
Lamp Lamp point_cloud_952.gif Correctly classified as Lamp
Chair Chair 406_point_cloud_mis_lamp.gif Rectified: Correctly classified as Chair, which was not the case in the vanilla Pointnet
Vase Lamp point_cloud_619.gif Failure Case: Still not able to classify correctly
Chair Chair point_cloud_1.gif Correctly classified as Chair